A mathematical programming based characterization of Nash equilibria of some constrained stochastic games

We consider two classes of constrained finite state-action stochastic games. First, we consider a two player nonzero sum single controller constrained stochastic game with both average and discounted cost criterion. We consider the same type of constraints as in [1], i.e., player 1 has subscription based constraints and player 2, who controls the transition probabilities, has realization based constraints which can also depend on the strategies of player 1. Next, we consider a N -player nonzero sum constrained stochastic game with independent state processes where each player has average cost criterion as discussed in [2]. We show that the stationary Nash equilibria of both classes of constrained games, which exists under strong Slater and irreducibility conditions [3], [2], has one to one correspondence with global minima of certain mathematical programs. In the single controller game if the constraints of player 2 do not depend on the strategies of the player 1, then the mathematical program reduces to the non-convex quadratic program. In two player independent state processes stochastic game if the constraints of a player do not depend on the strategies of another player, then the mathematical program reduces to a non-convex quadratic program. Computational algorithms for finding global minima of non-convex quadratic program exist [4], [5] and hence, one can compute Nash equilibria of these constrained stochastic games. Our results generalize some existing results for zero sum games [1], [6], [7]. ⋆ A portion of Section 3 (two player case) has been presented in the 8th International ISDG workshop at University of Padova, Italy on 21-23 July, 2011. Vikas Vikram Singh Industrial Engineering and Operations Research, Indian Institute of Technology Bombay, Mumbai 400076, India E-mail: vikas−[email protected] N. Hemachandra Industrial Engineering and Operations Research, Indian Institute of Technology Bombay, Mumbai 400076, India E-mail: [email protected] 2 Vikas Vikram Singh, N. Hemachandra